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Abstract — In this paper we introduce neuro-fuzzy based 
clustering approach for content based image retrieval using 
2D-wavelet transform(2D-DWT). Most of the image 
retrieval systems are still incapable of providing retrieval 
result with high retrieval accuracy and less computational 
complexity.To address this problem, we developed neural 
network -fuzzy logic cluster based approach for content 
based image retrieval using 2D-wavelet transform. The 
system performance improved by the learning and 
searching capability of the neural network combined with 
the fuzzy interpretation. This overcomes the vagueness and 
inconsistency due to human subjectivity. Multiresolution 
analysis using 2D-DWT can decompose the image into 
components at different scales, so that the coarest scale 
components carry the global approximation information 
while the finer scale components contain the detailed 
information. The empirical results show that the precision 
improved from 78% to 98% and average recall rate of 77% 
to 98% for the general purpose database size of 10000 
images compared with other existing approaches. 

Index Terms — Color and texture based image retrieval, 
clustering, 2D Wavelet transform, Neural network and 
Fuzzy logic 

I. INTRODUCTION 

With the explosive growth in image records and the 
rapid increase of computer, retrieving images from a 
large-scale image database becomes one of the most 
active research fields. To give all images text annotations 
manually is tedious and impractical and to automatically 
annotate an image is beyond current technology. Content- 
based image retrieval (CBIR) is a technique to retrieve 
images semantically relevant to the user's query from an 
image database. In content based approach, users provide 
the system with image examples to retrieve the images. It 
is based on automatically extracted visual features from 
an image such as color, texture, and shape. Despite 
tremendous improvement in content based retrieval,it still 
have many limitations. First, it is difficult for the users to 
specify the visual queries with the low level features. 
Second, low level image features cannot precisely 
describe user information needs .However the gap 
between these low-level features and high level semantic 
meanings usually leads to poor performance.[l][ll]. The 
process of grouping a set of objects or patterns into 
classes of similar objects is called clustering. Clustering is 
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a process that organizes a data set into a number of 
groups (clusters) such that patterns within a cluster are 
more similar to each other than patterns belonging to 
different clusters. In other words, clustering is an 
important technique for discovering the inherent structure 
in any given pattern set. It has been applied across many 
disciplines including engineering, statistics, psychology, 
sociology, astronomy, biology, business, medicine to 
name a few [10] [16]. There exists two categories for 
clustering task: hard and soft clustering. In hard 
clustering, each data object is assigned to exactly one 
cluster, while soft clustering is more desirable to let a data 
object be assigned several clusters partially. Hence, the 
soft clustering is also called neuro-fuzzy clustering 
[20][21]. Our goal of this paper is to provide 
methodology that concentrates on retrieving of images, 
defined by variety of clusters, in order to find the 
particular clusters. In this approach we integrated color- 
texture based image retrieval method with use of neuro- 
fuzzy system. From the initial returned images, users can 
select different clusters and retrieve from the database. 
Most of early CBIR systems perform retrieval based 
primarily on global features including IBM'S QBIC[22] 
and MIT'S Photobook[23].It is common that users 
accessing a CBIR system look for objects. Thus,the 
aforementioned systems are likely to fail, since a single 
feature computed for entire image content cannot 
sufficiently capture the important properties of the 
individual images. In this paper, we present neuro-fuzzy 
clustering technique to retrieve the images as clusters 
with human visual perception. The rest of the paper is 
organized as follows: Section II presents the previous 
related works in the area of fuzzy and neural network 
based image retrieval. Section III presents architecture 
developed to facilitate the proposed approach and the IV 
section we describe the color and V section texture 
feature based retrieval. Section VI experimental results 
and we presented conclusion in section VII. 

II .Related work 

Fagin[24] and Ortega et al [25] are pioneers who 
integrated fuzzy logic models into CBIR systems .They 
proposed algorithms to evaluate the fuzzy query, and 
showed the effectiveness through proven theorems and 
experimental results. Medasani and krishnapuram [14] 
proposed a fuzzy-based linguistic query in their CBIR 
system. Their membership query is formulated by a 
Gaussian mixture membership function and fuzzy 



47 



©2010 ACEEE 
DOI:01.ijsip.01.01.10 



^cACEEE 



ACEEE International Journal on Signal and Image Processing Vol. 1, No. 1, Jan 2010 



connectives. It provides natural and friendly interface to 
their system. Dubois et al [13] illustrated a fuzzy logic 
framework for a query by example scheme, where the 
users can give relevant or irrelevant examples together 
with their significance. Their framework shows the 
feasibility of using fuzzy-based relevance feedback in 
CBIR systems. Lin et al [26] designed a fuzzy logic CBIR 
system for finding textures. Their system provides users 
with linguistic and visual queries which are formulated by 
fuzzy triangular membership functions. The Lin's system 
also incorporates relevance feedback to improve the 
retrieval accuracy through iteratively modifying 
membership functions. Their fuzzy-based framework is 
simple, intuitive and effective in texture retrieval. It is 
worth mentioning that Lin's system as well as other fuzzy 
logic CBIR systems, does not retain each user's 
preference during retrieval, thus retrieval accuracy is 
impaired. Chih-Yi Chiu et al[ 12] framework is proposed 
to alleviate two problems in traditional CBIR systems, 
including the semantic gap and the perception 
subjectivity. Kulkarni et al [6], [17], [18] proposed a neuro - 
fuzzy technique for CBIR.lt is based on fuzzy 
interpretation of natural language, neural network 
learning and searching algorithms. Our proposed system 
based on neuro-fuzzy clustering with multiresolution 
analysis using wavelet transform capable of achieving 
better precision, average recall rate and classification rate. 

III. PROPOSED ARCHITECTURE 

The architecture developed to facilitate color and 
texture based searching is illustrated schematically in 
Fig. 2. The query processing and image matching is the 
main contribution of the architecture. Features of images 
such as color and texture are extracted from image and 
stored in the database. The query image is processed to 
compute color features as in section IV. The same query 
image is given as input image for texture based retrieval, 
the 2D-DWT is applied to query image, texture features 
of the query image and database image is stored in the 
database. Our proposed work takes the advantage of the 
membership degree to each feature, fuzzifying the neural 
net works output by means of membership function. The 
query to retrieve the images from database is prepared in 
terms of natural language such as mostly content, many 
content and few content of the some specific color. Fuzzy 
logic is used to define the query. We define nine colors 
that fall within the range of human perception. The feature 
representation set of colors and texture is {red, green, 
blue, white, black, yellow, orange, pink, 
purple,energy} .These ten colors have been used as input 
to the neural network and produces content type as 
output. Mostly, many, and few indicate the output. 
Accordingly the related similarity measure should 
proximity between two vectors, independently described 
by their respective features. 

Color is the most popularly used features in image 
retrieval and indexing. On the other hand, due to its 
inherent nature of inaccuracy in description of the same 
semantic content by different color quantization and /or 



by the uncertainty of human perception, it is important to 
capture this inaccuracy when defining the features. We 
apply fuzzy logic to the traditional color histogram to 
help capture this uncertainty in color indexing [2] [5]. 
We assume that any color is a fuzzy set. That means we 
will associate any color to a fuzzy functions, jU c : fl -> [0,1] 
and for any color c of the universe, fJ c (c) is the 
resemblance degree of the color c to the color c.The fuzzy 
model we define should follow the property that the 
resemblance degree decreases as inter-color distance 
increases. So we assume the particular color content for 
each image to mostly, many and few. In our model, the 
interpretation ranges of the values used are [0.9,1] for 
mostly,[0A,0.5] for many and [0.15,0.25] for few[9][ll]- 
[14]. 

A .Semantic rule M 

M is the semantic rule mapping from low-level feature U 
to the high level fuzzy semantics of T, that is to say, 
foruE U,t E T(x).lt assumes a value in [0,1] called 
degree of membership to U according to the linguistic 
value t.We can formally describe the task as sample set 

l( H >i) - ■ dn>h) K' n which }'i e U& = i, -n) 

represents color histogram and Jj(| = 1 ... .71) is the 
degree of membership. [17]Neural network is used as an 
adaptive retrieval system which incorporates learning 
capability into the network module where the network 
weights represent adaptability. This learning approach has 
several advantages over traditional retrieval approaches. 
It allows the retrieval system to solve the problem of 
fuzzy understanding of users' goals [7][15]-[17] 
We have adopted a Self-Organizing Tree Map (SOTM) 
and a Learning Vector Quantization (LVQ) which 
together were used to form a local data clustering [9] [15]. 
The SOTM works well for the combined color-texture 
features for the construction of unsupervised suitable 
clustering algorithm. It is more effective than the K- 
means and the self organizing feature map (SOM) 
algorithm particularly when the input space is high 
dimensionality. Thus, SOTM is chosen in our work to 
locate cluster centers in the high dimensional space of 
image features. To carry out SOTM/LVQ algorithms, we 
have given a set of training samples corresponding to 
retrieved images from a previous search operation 
[15]. This is used for approximating the distribution of the 
relevant samples which are associated with the training 
data and incorporated into radial basis function (RBF) 
network for ranking of an image database [29]. 
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Figure. 1 Proposed architecture 
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Figure.2. Architecture of the RBF netwok 

We denote this training data with two sets of vectors. 
Positive sample set(relevant images) X + = {x%, -X^} c: 
FP, and negative sample set(non-relevant images) X~ = 
{x\, —Xm\ c fiP.We assign each input vector in,X + into 
an SOTM algorithm to create the weight vectors, where 
as the vectors mX~ are used in the LQV algorithm for 
further adjustment of the weight vectors. SOTM 
algorithm is given as follows. [15] 

Step 1. Initialization. Choose the root nodes {uj}' =1 from 
the available set of input vectors {x^.An a random 
manner. 

Step 2. Similarity matching. Randomly select a new data 
point Xand find the best matching neuron at time step t 
by using the minimum-distance Eudlidean criterion: 

w j=Ejix(t)-wj||j=12 1 W 

Step 3. Updating. \f\\x{t) -W)|| <//(£),where //(£) is 
the hierarchy function used to control the levels of tree, 
then assign x{t) to the jth cluster,and adjust the synaptic 
weight vector according to the reinforced learning rule: 

Wj(t+ l) = vrjit) +a(t)\xlt) - vry(t)|, (2) 

Where a(t)is the learning rate, which decreases 
monotonically with time,0 < d(t) < 1. Else form a new 
subnode starting withx 

Step 4. Continuation. Continue with step 2 until no 
changes in the feature map. 

The SOTM algorithm obtains a new set of cluster centres 
Vq — {tii — t/c ■■■t/^}, where the number of centres K is 
controlled by the the function //(.£). In the 
experiment, H(t) was initialized by the norm of the 
training vectors in A + ,and was reduced linearly. 
Step 5. Cluster Modification .At this stage, the negative 
samples in,X™are used to tune the decision boundaries of 
the initial set l^j.We employ the antireinforced learning 
rule in the LVQ algorithm to perform this operation. The 
algorithm starts with randomly choosing input vector 
X m (t) from the training set {Xm}ui=l- Then, classify 
%i(£) to the node t^ if 

lkn£)-^ll <\\x m (t)-v k \\,v-k*i (3) 

apply the antireinforced learning rule to the 

corresponding cluster centers as 

tj(£ + 1) = tj(£) - 17(f) [J4(t)-tj(£)] (4) 

Where Tj{t) is the learning constant which decreases 
monotonically with the number of iterations £, < 
7](t) < 1. After several training data, the node vectors 
converge and the cluster modification process is 
complete. 

We assign each input vector v m as centre of the 
corresponding Gaussian kernel in the network.For the 



arbitrary input vector x,the output of the rath RBF unit is 
given by 

C„( X, \,> lr ] = ^_&*&S>> (5) 

^ m 

Where a m is a smoothing parameter defined as 

a m = 5mir J ||V ni -V i ||,M,2....M (6) 

with 5=0.5 being an overlapping factor. The estimated 

function output f(x) for x is then given as 



/fx)=3w=lQ n (X,V m aJ 



(7) 



TABLE I 

Neural network parameters for experiments 



Inputs 


Outputs 


Hidden 
Units 


Training 
Pairs 


77 


a 


Iterations 


10 


3 


5 


307 


0.7 


0.2 


50 



IV. Color Feature Based Retrieval 

Image retrieval systems have implemented one or more 
color features, for example, dominant colors, color 
moments, color sets, color coherence vector, color 
correlogram and autocorrlogram. The good color space 
for image retrieval system should preserve the perceived 
color differences. In other words, the numerical Euclidean 
difference should approximate the human perceived 
difference [5] 

1. Color Conversion 

In order to use a good color space for a specific 
application, color conversion is needed between color 
spaces.Red,Green,Blue (RGB),Hue Saturation Value 
(HSV). 

A. RGB to HSV Conversion 

In Fig.3 Obtainable HSV colors lie within a triangle 
whose vertices are defined by the three primary colors in 
RGB space[l] 



ki i-i a: 




Figure. 3. RGB to HSV Conversion 

The hue of the point P is the measured angle between the 
line connecting P to the triangle centre and line 
connecting RED point to the triangle centre. 
The saturation of the point P is the distance between P 
and triangle centre. The value (intensity) of the point P is 
represented as height on a line perpendicular to the 
triangle and passing through its centre. The greyscale 
points are situated into the same line and the conversion 
formula is as follows 



H = Cosr 1 {-^ 



3[(F-G)+:F-B)] 



|(F-G)2+;F-B)(C-B> 



S=l- m % w [mm(R,G,B)] 



(8) 



(9) 
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V = *(R + G + B) (10) 

B. HSV to RGB Conversion 

Conversion from HSV space to RGB space is more 
complex and given to the nature of the hue information, 
we will have a different formula for each sector of the 
color triangle. [1] 
Red-Green sector for 0°<H<120° 



_Scos_H_[ 
-H)l 



h = l(i-s) 
g=i-(r + h) 

Green-Blue sector for 120° <H< 240° 

r = l(i-s) 



9 
b- 



- ill _l _ScosH\ 
' si 1 ^C0S6(-H)I 

i-(r+b) 



Blue-Red sector for 240°<H<360° 

r= i-(r+fe) 



g=ta-s) 



h = \\i 



ScosH 



CO&6Q-H) 



— I 



(11) 

(12) 
(13) 

(14) 
(15) 
(16) 

(17) 
(18) 

(19) 



V. Texture Based Image Retrieval 

Texture is another important property of images. Various 
texture representations have been investigated in pattern 
recognition and computer vision. Basically, texture 
representation methods can be classified into two 
categories: structural and statistical. Structural methods, 
including morphological operator and adjacency graph, 
describe texture by identifying structural primitives and 
their placement rules. They tend to be most effective 
when applied to textures that are very regular. Statistical 
methods, including Fourier power spectra, co-occurrence 
matrices, shift-invariant principal component 
analysis(SPCA),Tamura feature [2 8], wold 

decomposition, Markov random field, fractal model, and 
multire solution filtering techniques such as Gabor[26] 
and wavelet transform[3],[27],[28]. 
Since image is typically a two dimensional signal, a 2D 
equivalent of the DWT is performed. This is achieved by 
first applying the L and H filters to the lines of samples, 
row by row, then refiltering the output to the columns by 
the same filters. As the result, the image shown in Fig.3 is 
divided into 4 subbands, LL, LH, HL and HH as depicted 
in Fig.4. The LL sub band contains the low pass 
information of horizontal, vertical and diagonal 
orientation. The LL sub band provides a half sized 
version of input image which can be transformed again to 
have more levels of resolution. Generally, an image is 
partitioned into L resolution levels by applying the 2D 
DWT(L-l)times[3],[27] 

By wavelet transform, we mean the decomposition of an 
image with family of real orthogonal bases 
Iprrmix) obtained through translation and dilation of a 
kernel function \}){x) known as mother wavelet 



$hm(x) = 2 tifKT^x-n) 



(20) 



Where m and n are integers. Due to the orthonormal 
property,the wavelet coefficients of a signal f(x) can be 
easily computed via 

C ™=fXfW4mn {X)dX (21) 

and the synthesis formula 

f\X) — Zj/nn^mnVmnW (22) 

can be used to recover f(x) from its wavelet 
coefficients 




Fig.4. Orginal image Fig.5.0ne level decomposition 

To construct the mother wavelet lp(x), we may first 
determine a scaling function (f)(x), which satisfies the 
two-scale difference equation 

(fy(x) = V2" Y k h(k)4>(2x - k) (23) 

Then the wavelet kernel lp(x) is related to the scaling 
function via 

ip(x) = V2" Z k g(k)(f>(2x - k) (24) 

where 

g(k) = (-i)*ft(i -k) (25) 

The coefficients h(k) in (23) have to several conditions 
for the set of basis wavelet functions in (21) to be unique, 
orthonormal, and have a certain degree of regularity. 
The coefficients h(k) and g(k) play a very crucial role in 
a given discrete wavelet transform.To perform the 
wavelet transform does not require the explicit forms of 
(p(x)and */j(x)but only depends on ft(fc)andg(k). 
Consider a J-level wavelet decomposition which can be 
written as 

fo(x) = T lk c ,k4 k(x) 

= 5V041 /«£j41 *(*) + 4*^41 k 4)+l *(*)) ( 26 ) 
Where coefficients CQ k are given and coefficients Cj+i n 
and (?j+i n at scale/ + 1 are related to the coefficients Cj k 
at scale / via 
fyin=2kf)kh(k-m (27) 

^m = ^ k Cj k g{k-2n) (28) 

Where < /' < /.Thus,(27) and (28) provides a recursive 
algorithm for wavelet decomposition through ft(fc) and 
_g(fc),and the coefficients Cj n for a low resolution 
component (fij k (X).By using a similar approach, we can 
derive a recursive algorithm for function synthesis based 
on its wavelet coefficients dj n < j < J and Cj n 
Cj k = InCj+ ln h{k - 2n) + En£j4in5(fe " 2 ™) ( 29 ) 

It is convenient to view the decomposition (8) as passing 
a signal Cj k through a pair of filters H and G with impulse 
responses h[7l) and g (n) and down sampling the filtered 
signals by two (dropping every other sample), where 
h{7l) and^ln) are defined as 
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h\ri) = h(-ri),g\n) = g(-n). 

The pair of filters H and G correspond to the half band 
low pass and high pass filters respectively, and are called 
the quadrature mirror filters in the signal processing 
literature [5] 

The reconstruction procedure is implemented by up 
sampling the sub signals C^\ n and ^ 41 n and filtering 
with h(n) and g(Tt) Respectively, and adding these two 
filtered signals together.Usually the signal decomposition 
scheme is performed recursively to the output of lowpass 
filter h in). 

The wavelet packet basis functions {H^}^ = can be 
generated from a given function Wq as follows 
W 2n (X) = V2 S/cWWMn (2* - k) (30) 



W 2M1 (X) = j2Z k g(k)]A^(2x-k) 



(31) 



Where the function Hq(x) can be identified with the 
scaling function (fy and Hi with the mother wavelet 
^.Then,the wavelet packet bases can be defined to be the 
collection of orthonormal bases composed of functions of 
the form M^(2^x-fc), where /, keZ.neN- Each element 
is determined by a subset of the indices: a scaling 
parameter /, a localization parameter k, and an oscillation 
parameter n. 

The 2D wavelet (or wavelet packet) basis functions can 
be expressed by tensor product of two 1-D wavelet (or 
wavelet packet) basis functions along the horizontal and 
vertical directions. The corresponding 2-D filter 
coefficients can be expressed as 

h LL {k, I) = h(k)h(ll h LH (k, I) = h(k)g(l), (32) 



h HL (k,l)=g(kMin HH (kd)=g(k)ga) 



(33) 



Where the first and second subscripts in (33) and (34) 
denotes the low pass and highpass filtering characteristics 
in the x-and y-directions respectively [3]. 

VI.Results and discussion 

To evaluate the performance of our image retrieval 
clustering algorithm, the proposed framework is tested 
with a general -purpose image database of about 10000 
images each size 256x256 approximately 1000 categories 
from COREL. In our system we have selected categories 
such as roses, buses, elephants, lions, and sunflowers etc. 
The system supports both image retrieval as well as 
clustering. For each experiment we first selected one 
image as a query image randomly from the database and 
then obtained the retrieved images. Then we selected one 
or more images as clusters from the retrieved images. The 
selected images (clusters) were used as query images for 
next step to retrieve the particular clusters. The wide 
range of queries which could be submitted as a result of 
different query images also possible. The simulation 
results of the image retrieval is shown in Fig. 6. Retrieval 
of Clusters 1 shown in Fig. 7, Retrieval of Clusters 2 
shown in Fig. 8, Retrieval of Clusters 3 shown in Fig. 9, 
Retrieval of Clusters 4 shown in Fig. 10. Retrieval of 



Clusters 5 shown in Fig. 11. Our system has been tested 
with the average recall rate (AVRR), and precision 
measures[l]used the database with 10000 real world 
images 

(31) 
(32) 
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AVRR= ^ 
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A=Number of retrieved images, 
B=Number of relevant images 
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Figure. 10. Cluster 4 retrieval. 
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Figure. 1 1 Cluster 5 retrieval. 



TABLE II 

Precision for various approaches 



Approach 


Average recall rate 


Number of query images 


1 


2 


3 


4 


5 


NN 


0.7713 


0.772 


0.773 


0.7743 


0.7855 


RFM 


0.6910 


0.699 


0.698 


0.6994 


0.6992 


Fuzzy 


0.7874 


0.780 


0.772 


0.7812 


0.7886 


Proposed 


0.98 


0.98 


0.98 


0.98 


0.98 



Table II provides a detailed comparison of precision obtained 
for the top five queries considered using NN,RFM,Fuzzy and 
proposed approach. Precision performance improved upto 21% 
compared to NN and upto24% compared to RFM and upto 20% 
compared to Fuzzy approach on query image no. Land almost 
precision for all other query images. For the top five queries 
computed and tabulated in Table. II and other 10 query images 
shown in Fig. 12. 



TABLE III 
AVERAGE RECALL RATE FOR VARIOUS APPROACHES 



Approach 


Precision 


Number of query images 


1 


2 


3 


4 


5 


NN 


0.763 


0.764 


0.7636 


0.7635 


0.7655 


RFM 


0.711 


0.729 


0.7286 


0.7294 


0.7292 


Fuzzy 


0.719 


0.718 


0.7125 


0.7125 


0.1189 


Proposed 


0.98 


0.98 


0.98 


0.98 


0.98 



Table III provides a detailed comparison of Average 
recall rate obtained for the top five queries considered using 
NN,RFM,Fuzzy and proposed approach. Precision performance 
improved upto 22% compared to NN and upto27% compared to 
RFM and upto 27% compared to Fuzzy approach on query 
image no. 1 .and almost precision for all other query images. For 
the top five queries computed and tabulated in Table. Ill and 
other 10 query images shown in Fig. 13. 
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Figure. 12 Precision for 10 different query images 
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Figure. 13 Average recall rate for 10 different query images 

VI. CONCLUSION 
For the sake of alleviating the semantic and perception 
subjectivity problems, a CBIR system should well 
characteristize a mapping from image features to human 
concepts, and effectively captures the user's preference 
on retrieval by means of clustering. The neuro-fuzzy 
system with clustering can provide a flexible mapping 
from low level features to high level human concepts. 
The main contribution of this works is in building various 
features required by a working system for efficient and 
effective retrieval of images. This proposed approach 
greatly reduces user's effort and captures the user's 
information needed more precisely. Furthermore, the 
effectiveness of the proposed approach have been 
validated by a large number of experiments. Our 
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proposed approach improves the precision from 79% to 
98 %, compared with NN approach,71% to 98% 
compared with RFM approach and 78% to 98% 
compared with Fuzzy approach .And almost high same 
average recall rate compared with NN approach, RFM 
approach and Fuzzy approach. Also our proposed 
approach retrieves the images as clusters with high 
human visual perception and more classification rate. 
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